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Abstract 

Three questions about developmental assessment centers are addressed: (His assessment 
center feedback accepted and acted on by participants?; ( 2) Does this feedback improve 
managerial performance?; (3) What Is the utility of assessment assessment centers fcr career 
development? Participants (a s 1 02) appeared to accept and act upon the assessment feedback. 
Poor performers followed the recommendations to the same extent as high performers, and the 
participants believed they could Improve their managerial performance by following the 
recommendations. After receiving feedback, participants were able to Improve their performance 
on a measure of managerial effectiveness. Utility analysis Indicated that the gain In productivity 
from the assessment far outweighed the costs. 



ERIC 



3 



Assessment Centers 
3 

The Utility of Assessment Centers for Career Development 
During the past 30 years, assessment centers have become an increasingly popular 
method for evaluating employees and potential employees In work organizations. Geugler , 
Rosenthal, Thornton and Bentson ( 1 987) estimate that over 2,000 organizations are 
currently usinn some type of assessment center program. Recently, many organizations also 
have been implementing newly-designed career development programs. A promising 
development in this area, according to Wexlev ( 1 984), is the career development assessment 
center. Developmental assessment centers are designed to provide an evaluation of an 
individual's potential to perform at a higher level, and seem to have considerable appeal to 
participants and management. Developmental centers may integrate career planning into the 
human resources system by providing feedback on managerial strengths and weaknesses, and 
by recommending developmental activities such as training. By having their abilities to 
perform managerial activities evaluated, individuals who are not suited for management 
positions could self-select themselves out of the promotional process, according to Wexley. 

The purpose of the present study was to evaluate an assessment center which was part of 
a management development program for state government employees. The study attempted to 
answer three questions about the assessment center: ( 1 ) is the feedback from the assessment 
center accepted and acted on by the participants?; ( 2) Does participation in the assessment 
center and feedback session improve participants* managerial performance?; and, ( 3) what is 
the economic utility of the assessment center for career development? 

Method 

Participants were 1 02 middle- managers in state government. The sample included 39 
female and 21 black participants. Participation in the program was voluntary. An Independent 
sample of 20 middle- managers volunteered to serve as the control group in this stud/. These 
managers were eligible to participate, but had not yet been assigned to the program. 
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The dimensions and the exercises used in the assessment center were based on an extensive 
Job analysis of middle-manager positions ( Hauensteln. 1 985). The job analysis identified 1 0 
dimensions as critical for successful Job performance: Integrity, Problem Detection. Initiative, 
Persuasiveness. Time Sensitivity. Sensitivity. Group Leadership. Organization and Planning. 
Written Communication . and Oral Communication. Job analysis results were also used in the 
design of the tasks, settings, and scoring criteria for the following assessment center exercises.- 
( 1 ) Patterned interview; (2) Meeting Exercise (small leederless group discussion); (3) Policy 
Analysis Presentation Exercise (oral presentation); ( 4) Policy Implementation Plan 
(development of a written plan for implementing policy changes); (5) Written Situational Test; 
( 6) Policy Discussion Exercise ( large leaderless group discussion); ( 7) Performance Counseling 
Exercise; and. (8) In-Basket Exercise. 

At the end of each exercise, from one to three assessors rated the participant's performance 
on the relevant dimensions using a five-point scale ( 1= less than acceptable; 5 = outstanding). 
The participants also received scores on three standardized psychological instruments: Leadership 
Opinion Questionnaire (LOO) (Fleishman, 1960); Leader Behavior Description Questionnaire 
(LBDQ) (Stogdlll. 1963); and. Personality Research Form (PRF) (Jackson. 1974). After the 
assessment was completed, the assessors met with the center director to reach consensus decisions 
on dimension ratings for each participant. 

Based on the assessment center ratings, the center director l£„tlf led the managerial 
strengths and weaknesses of each participant. In general, a dimension rating of less than three 
indicated a weakness in that dimension. On-the-job performance ratings ( self, superior, and 
subordinate) er,d standardized test scores also were considered by the director in determining 
managerial strengths and weaknesses, in a feedback session, these strengths and weaknesses were 
presented to the Individual participants. The director also recommended a plan of study indicating 
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which in-house management training courses each participant should take. The assessment 
results were confidential and made available only to the individual participants. 
Design and Pmrartirft 

To measure participant reactions to the assessment, a 26-ltem questionnaire based on the 
feedback model of llgen, Fisher, and Taylor ( 1 979) was developed. This model indicates that 
feedback affects behavior by: ( 1 ) the way it is perceived, (2) its acceptance by the recipient, and 
( 3) the willingness of the recipient to respond to feedback. Using this questionnaire, the 
participants' rated their perceptions the feedback process on a 5-pofnt scale, with 5 indicating the 
most favorable response. 

An index of the match between the courses recommended in the feedback sessions and the plan 
of study was used to deter m ine the extent to which the plan of stud/ decisions were based on the 
feedback. This index Is the percentage of recommended courses that were recorded on the 
respective plans of study for each participant. The participants could take all or none of the 
recommended courses; however, they were required to complete their plans of study within two 
years. The courses were generally one to two weeks 1 length. 

The responses from the reaction questionnaire, the match index, and the assessment center 
ratings were used to analyze the participants' reaction to and acceptance of the feedback. These 
measures were used in a correlational analysis as the following three variables: (A) responses to 
the reaction questionnaire as the measure of acceptance of the assessment center , ( B ) the match 
index as the measure of the extent to which recommendations are acted on, and ( 0 the assessment 
center ratings as the measure of performance. 

To determine if participation in the development center Improved the managerial 
performance of the subjects, a subsample of 33 participants volunteered to participate as the 
treatment group in a pre- and posttest design. A control group of 20 state government managers 
was selected to receive the pre- and posttests. Managerial effectiveness was measured with eleven 
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items from the written situational test exercise as a pretest measure, and a parallel form as the 
posttest measure. The written situational test was based on situational interview methodology 
developed by Latham, Saarl, Pursell, and Campion ( 1980). The treatment group had completed 
this exercise as part of the normal assessment process. The pre- and posttest scores of the two 
groups were analyzed as the dependent variable in a two-way analysis of variance, as 
recommended by Cook and Campbell ( 1 976, p. 25 1 ). To determine the effectiveness cf the 
assessment and feedback process, an effect size was calculated using the formula employed by 
Burke and Day ( 1 986). Effect size, the metric commonly used in meta-analysis, is the 
normalized difference between the treatment group and the control group. 

To determine the results of the program in terms of oconomlc utility, rating scales were 
developed based on the Cesclo- Ramos Estimate of Performance in Dollars (CREPID) (Casclo, 
1 982). This model was used to estimate the standard deviation in dollars (SDy) for job 
performance of the participants. A subsample of 25 assessment center participants rated the 
seven most critical management functions on the CREPID rating scales. The functions were rated 
in terms of time/frequency, importance, consequence of error and level of difficulty. The obtained 
SDy, effect size, and assessment center costs were entered Into the utility equation, as described 
by Landy and Farr ( 1 983, p.268). 

Results 

The results indicate that attitudes toward the assessment center and the feedback were 
generally favorable. Overall , participants thought that they would be better managers as a result 
of their participation in the assessment center. 1 he participants responded that they planned to 
base their plan of study on the feedback (fl, * 4.29. $D_ = .57). and that the recommendations from 
the assessment center were worthwhile (tl = 4.28, SD. = .53). The respondents believed that they 
will be better managers by improving themselves based on the feedback (tl = 4.20. $D_ = .60), 
and that the feedback was valuable for their personal development (d =4.33, $D_ = .60). The 
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lowest rated responses concerned the ability of the participants to refuse to attend the assessment 
centsr without negative consequences (tl= 3.35, S£ = .86), and the amount of Information 
provided to the participants prior to their attending (d= 3.54, 5[>= 1.06). Only 39 
participants ( 50* ) felt that they could have refused to attend the assessment center without 
negative consequences. 

Plan of study data were available from 7 1 participants. A mean of 7.8 courses was 
recommended to the participants In the feedback session, and a mean of o.3 courses was actually 
included on the participants' plans of study. The index of the match was calculated for each 
participant, and on the average, 8 1 % of the courses recommended were included on the plans of 
study. The match index ranged from 25* to 1 00* , with a standard deviation of 1 6*. 

The relationship between assessment center performance and the extent the recommendations 
were followed was not significant. Low performers in the assessment center followed the 
recommendations to the same extent as did high performers. Also, there was no relationship 
between the level of acceptance of the feedback and assessment center performance. Low 
performers reacted to the feedback as well as high performers did. Contrary to the expectations, 
however , there was no relationship between the level of acceptance of the feedback , and the extent 
to which the recommendations were acted cr,. Participants who responded less favorably to the 
feedback followed the recommendations as closely as those who responded favorably. 

The results of a repeated-measures analysis of variance indicate that the participants were 
able to improve their scores on a measure of managerial effectiveness after participating in the 
development center. Table 1 reports descriptive statistics for the oretest and posttest scores for 
the treatment and control groups. Table 2 is the ANOVA table for the written situational test 
scores. Group membership served as the between-subjects factor , and the time of testing as the 
wlthln-subjects factor. The effects of the wlthln-subjects factor were significant, indicating that 

0 

the difference between the pre- and posttest scores of the two groups was significant. The 
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interaction between group membership end time of testing was also significant (p. i .000 1 ), 
indicating that the effects of the treatment were significant ( Cook & Campbell, 1 976 . p. 251). 



insert Table 1 and 2 about here 



The effect size for the assessment and feedback process was .64, with a wlthln-group 
standard deviation of 5.53. Thus, if a person In the control group were to go through the 
assessment and feedback, his or her managerial performance would be expected to improve .64 
standard deviation units ( Burke & Da/, 1 986). 

Using the CREPiD technique (Casclo, 1 982), the standard deviation of performance in dollars 
was estimated to be $ 1 3 ,925. Based on the financial records of the program , the cost of 
conducting the assessment center was estimated to oe $ 41 3 per participant. By substituting the 
SDy, the effect size, and the cost component into the aquation described by Land/ and Ferr 
(1983), the utility of the assessment center program was estimated to be $8,499 per participant 
for a single year. For the 1 02 state government managers who participated in the program, the 
gain in productivity for oneyear was $868,898. Pedollar value ofthe improvement in 
managerial performance as the result of assessment center participation, therefore, was much 
greater than the cost of conducting the assessment centers. 

Discussion 

The responses to the reaction questionnaire Indicate a positive perception of the feedback, and 
a fairly high level of acceptance by the participants. Participants agree that their knowledge of 
their managerial skills was increased as a result of the feedback , and that the feedback provided 
new insights into their managerial behavior. The results of the match index show that the 
participants followed the recommendations from the feedback to a fairly high extent. 
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For a developmental assessment center , the acceptance of (he feedback by the participants is a 
critical factor in determining the effectiveness of the program. Low performers in this 
assessment center reacted to the feedback as well as high performers. Low performers also 
followed the recommendations to the same extent as the high performers. Participants who 
responded less favorably to the feedback followed the recommendations as closely as those who 
responded favorably. The feedback from this assessment center program , therefore, appears to 
meet tne criteria of effectiveness set by the llgen st al. ( 1 979) model 

With the measure of managerial effectiveness, the difference between the the control and 
treatment groups cn the posttest was significant, with the treatment group scoring higher. The 
difference between the two groups on the pretest was also significant, with the control group 
scoring higher. Cook and Campbell ( 1 976) discuss the Interpretation of outcomes of this type, 
where the low scoring pretest group ( the treatment group) overtakes the high scoring control 
group. These results are more interpretable than other outcomes of the non-equivalent control 
group design, accord ng to Cook and Campbell. They can be interpreted as an indication that, as a 
result of the treatment, the treatment group significantly improved their scores on the posttest. 
The possibility exists, however, that the effects of local history are a source of invalidity in this 
stud/. It is not known if any of the subjects participated in developmental activities between the 
time of the pretest and the posttest. 

Because the feedback was specific and behavlorally based, the participants apparently were 
able to apply misinformation when responding to items on the measure of managerial 
effectiveness. Although one cannot he certain that this learning will transfer to the work place, 
there Is sone evidence of a relationship between what employees said they would do in hypothetical 
situations, and what supervisors and peers observed them doing on the Job ( Latham & Saarl, 
1984). 
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From a review of the literature on the effects of feedback on perfcr mance, Landy and Ferr 
( 1 983) concluded that the effect size of evaluation and feedback was 60. According to Cohen 
(1977), an effect size of .64 is near the midpoint between a medium and a large effect. By these 
standards, the effect of the assessment center feedback on managerial performance in this stud/ is 
fairly large and of practical significance. 

In the present stud/ , the obtained SOy Is 39* of annual salary. Casclo and Ramos ( 1 986) 
found that their standard deviations across five )ob classes varied from 40* to 50* of the 
average actual wage. The estimate of the standard deviation of performance In dollars in the 
present study, therefore, appears to be conservative. LenoV end Farr ( 1 983) report that *h8 
utility of feedback from a performance appraisal system was $ 1 1 ,300 per manager. Therefore, 
the estimate of utility In the present study ( $8,499 per participant) also appears to be 
conservative. The results of the utility analysis indicate that assessment centers can be a cost 
effective method for developing managers. The estimated gain In productivity from assessment 
center participation far outweighs the cost of conducting the centers. 

In summary, it appears that assessment center technology can be a powerful tool for 
developing managers. One other study was found which Investigated the effects of assessment 
center feedback on subsequent performance. Barber (cited in Thornton & Byham , 1 932. p. 329), 
found that participants who received feedback from the assessment scored higher on a criterion 
measure based on leadership and academic performance. A control group received significantly 
lower scores on overal I performance ratings. The results of Berber's study and the present study 
provide some evidence of the effectiveness of assessment feedback In improving the managerial 
performance of the participants. More research is necessary before any definite conclusions can 
be drawn about tne efficacy of this process. More research also Is needed on the procedures used to 
validate assessment centers (e.g. , Sacket, 1 987). 
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Table I 

Descriptive Statistics for Pretest and Posttest Scn res on Written Situational Test for anntral and 
Treatment Groups 



a 




£1 




Std. 
Error 


Range 


Pretest 20 
(Control) 




21.40 


$.56 


1.24 


11.0-30.5 


Posttest 20 
(Control) 




20.22 


5.03 


1.12 


12.0-34.0 


Pretest 33 
(Treat.) 




15.09 


5.54 


0.97 


6.0-28.0 


Posttest 33 
(Treat.) 




23.77 


5.75 


1.00 


12.0-42.0 


NQk Std. Error = standard error of the mean. Control = control group, Treat. 


■ treatment group. 


Table 2 












Repeated- Measures Analysis of Variance Table far Written Situational Test Scores 




Source 


df 


Sum of 
Squares 


Mean 
Squares 


F-test 


a value 


Between subjects 
Group 

Subjects (groups) 


1 

51 


47.48 
2163.54 


47.48 
42.42 


1.12 


.295 


Within subjects 
Time 


1 


652.54 


652.54 


35.15 


.0001 


Group x Time 


1 


604.94 


604.94 


32.59 


.0001 


Subjects (Group x Time) 


51 


946.77 


18 56 
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